Making more active use of a secondary storage system

ABSTRACT

Techniques are disclosed which allow a secondary storage system to provide data to non-production workloads in conjunction with performing data backup and protection tasks. As disclosed, a secondary storage system exposes backup data stored by the secondary storage system to other workloads, such as test and development applications, data analytics, etc. These non-production workloads can run at the same time the secondary storage system provides backup services to a primary storage system. This consolidation eliminates the need for an enterprise to deploy separate storage clusters for analytics, test and development applications, etc. and eliminates unnecessary copies of data.

CROSS REFERENCE TO OTHER APPLICATIONS

This application is a continuation of co-pending U.S. patent applicationSer. No. 14/309,869 entitled MAKING MORE ACTIVE USE OF A SECONDARYSTORAGE SYSTEM filed Jun. 19, 2014 which is incorporated herein byreference for all purposes.

BACKGROUND OF THE INVENTION Field of the Invention

Embodiments described herein relate to a secondary storage system thatsupports multiple types of workloads. More specifically, embodimentsdescribed herein provide a secondary storage system that can runnon-production workloads while also providing data backup or protectionto a primary storage system.

Description of the Related Art

Workloads on a computer system are a way of characterizing the demand onthe resources of a computer system due to a particular task running onthe computer. For example, some types of workloads place extreme demandson the CPU while other types of workloads place extreme demands on thedisk subsystem. Often, workloads that place extreme demands on the disksubsystem are those that require a high number of random I/O operations.These operations thus cause the throughput of the disk system to dropsubstantially, making the system almost unusable, especially in themidst of other workloads attempting to use the system.

A primary storage system usually stores data for enterpriseapplications. For example, a primary storage system may store data for acluster of physical computing systems or virtual machine instancesrunning on a physical server system. The primary storage system operatesto provide fast, high-availability storage for the enterpriseapplications.

A secondary storage system is used to provide data backup and protectionservices for the primary storage systems supporting the enterprise.Secondary storage systems are often used only for data protectionpurposes, despite the fact that they may provide powerful computingpower. This occurs, in part, because secondary storage systems areoptimized for serial I/O operations used to capture data from theprimary storage system. In addition to storage systems, an enterprisemay deploy a variety of other systems or appliances to performnon-production tasks, such as business analytics or test and developmentclusters to develop, install, configure, and deploy enterprise softwareapplications. A secondary storage system optimized for serial I/Ooperations is ill-suited to perform these tasks. As a result, anenterprise may deploy one system that provides the data backup andprotection services for their primary storage systems and deploy othersystems to run test and development applications, analytics, etc.However, dedicating numerous separate secondary systems to eachindividual workload becomes inefficient and wasteful. At a minimum, itleads to a need for more physical systems, as well as to duplicate data,creating a maintenance problem.

SUMMARY OF THE INVENTION

One embodiment of the invention includes a method for providing accessto data stored in a secondary storage system. This method may generallyinclude receiving data from a primary storage system to store in thesecondary storage system. The received data provides a backup of datastored by the primary storage system. This method may generally includegenerating at least a first view of the received data and generating,from the first view, at least a second view. The second view exposes thereceived data to a client system according to a first protocol. Inresponse to receiving a read request from the client system for dataexposed by the second view, data from the second view is retrieved andreturned to the requesting client system.

In a particular embodiment, in response to a request from the clientsystem to perform a write operation to an element of the received dataexposed by the second view, the method may further include performingthe write operation to the requested element of data accessed via thesecond view without modifying a corresponding element of the receiveddata accessed via the first view. Additionally, the second view maycorrespond to a state of data in the primary storage system at a givenpoint in time, as captured by a backup stored by the secondary thesecondary storage system. Further, the first view and the second viewmay each have an associated tree data structure having a plurality ofnodes. Leaf nodes of the tree data structures store metadata indicatinga location of data exposed by the views. Further still, the step ofgenerating the second view may itself include copying a root node of thetree data structure associated with the first view.

Another embodiment includes a secondary storage system which includesone or more computing nodes, where each node has at least one computerprocessor, a memory, and one or more local storage devices and whereineach node hosts an instance of one or more applications which provideaccess to data stored in the storage devices by performing an operation.This operation itself may generally include receiving data from aprimary storage system to store in the secondary storage system. Thereceived data provides a backup of data stored by the primary storagesystem. This method may generally include generating at least a firstview of the received data and generating, from the first view, at leasta second view. The second view exposes the received data to a clientsystem according to a first protocol. In response to receiving a readrequest from the client system for data exposed by the second view, datafrom the second view is retrieved and returned to the requesting clientsystem.

Other embodiments include, without limitation, a computer-readablemedium that includes instructions that enable a processing unit toimplement one or more aspects of the disclosed methods as well as asystem having a processor, memory, and application programs configuredto implement one or more aspects of the disclosed methods.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 depicts an example secondary storage system, according to oneembodiment.

FIG. 2 depicts views of primary system on secondary storage system,according to one embodiment.

FIG. 3 depicts the interfaces for collecting and externally exposing theviews, according to one embodiment.

FIG. 4 depicts namespaces for different types of views, according to oneembodiment.

FIG. 5 depicts a process for creating a view, according to oneembodiment.

FIG. 6 depicts a process for creating a snapshot, according to oneembodiment.

FIG. 7 depicts a snaptree formed from (k,v) pairs managed by a scribeprocess, according to one embodiment.

FIG. 8 depicts a more detailed view of the snaptree and BLOBs, accordingto one embodiment.

FIG. 9 depicts a process for accessing the snaptree to service a datarequest, according to one embodiment.

FIG. 10 depicts an example system for a secondary storage server,according to one embodiment.

FIG. 11 depicts an arrangement of a production system hosting one ormore enterprise applications, a test and development system, and asecondary storage system according to one embodiment.

DETAILED DESCRIPTION

Embodiments presented herein provide an architecture for a secondarystorage system that can provide non-production workloads with access todata storage while also performing data protection tasks. For example,embodiments described below provide a secondary storage system that canexpose backup data to test and development applications, data analytics,etc. These applications can run at the same time the secondary storagesystem takes backups of a production system. This consolidationeliminates the need for an enterprise to deploy separate storageclusters for analytics, test and development applications, etc., andeliminates unnecessary copies of data. Doing so can significantly reducethe costs and management overhead needed to run the non-productionworkloads within an enterprise.

Further, embodiments presented herein isolate data protection relatedtasks from other active workloads. For example, as described below, asnapshot of a primary system can be created and exposed to anon-production cluster (such as a test and development or analyticscluster). Doing so allows the non-production workload to operate on alocal view of the data provided by the secondary storage system, withoutlimiting or interfering with the ability of an enterprise to use thesecondary storage system for the primary purpose—to capture data from aprimary storage system and to restore data to a primary storage systemif needed. Further, embodiments of a secondary storage system describedherein have an architecture with I/O capabilities needed by thenon-production workloads to operate efficiently and effectively.

In one embodiment, the secondary storage system is a set of nodes thatcaptures views, which capture backup data from the primary storagesystem. A current view generally corresponds to a (near) real-timebackup state of a primary system data store (e.g., a backup of thestorage of a virtual machine cluster, a database, or just a bunch ofdisks providing storage to an enterprise application). A snapshotrepresents a backup state of a primary storage a particular point intime. That is, each snapshot provides a state of data used by a primarysystem, which can be restored to the primary storage system if needed.Similarly, a snapshot can be exposed to a non-production workload, or aclone of a snapshot can be created should a non-production workload needto write to the snapshot without interfering with the original snapshot.

In one embodiment, secondary storage system organizes backup data for asnapshot in a tree structure, referred to as a snaptree. Each leaf nodeof the snaptree includes a key-value pair describing file systemmetadata. As described below, a snapshot of a view can be created bycloning the root node of that view. The structure of a snaptree allowssnapshots that can be created in (near) real-time and that can bequickly accessed and/or modified by a user of a development system useror business analytics user, etc.

FIG. 1 illustrates an example of a secondary storage system 100,according to one embodiment. As shown, the secondary storage system 100includes a plurality of servers 102,104 each connected to a network 106by network interface 128, 138, respectively. Each server 102, 104includes a plurality of storage units such as hard disk drives (HDDs)108, 110, 114, 116 for large amounts of storage and solid state drives(SSDs) 112, 118 for persistent storage with fast access. In oneembodiment, each server 102, 104, runs an operating system 126, 136 suchas a distribution of the Linux® operation system and supports a filesystem 124, 134 such as Ext4FS. As shown, each server 102, 104 executesan instance of a bridge process 120, 130 and an instance of a datagathering process 122, 132. As described in greater detail below, theseprocesses carry out the functions, including the backup functions, ofthe secondary storage system 100.

As noted, a view exposes a collection of data backed up from a primarysystem. For example, FIG. 2 depicts three primary systems, labeled P1202, P2 204, and P3 206 and a secondary storage system S 208. Primarysystems 202, 204, and 206 each provide a set of computing systems andapplications with data backed-up by the secondary storage system 208.Examples of primary systems include, e.g., P1 202 running the a databaseapplication, primary system P2 204 running an OpenStack cluster andprimary system P3 206 running a VMware cluster. As known, OpenStackprovides a collection of open-source software projects used to provide acloud-computing platform, including tools for managing processing,storage, and networking resources throughout a data center. And VMwareprovides commercial software products for managing a cluster of virtualmachine instances running on physical system.

In one embodiment, the secondary storage system 208 creates a view 210,212, 214 for each of primary system 202, 204, and 206. Each view exposesbackup data captured by data gathering process 122, 132 from therelevant primary system. That is, data gathering process 122, 132obtains data specified for backup from the primary systems 202, 204, and206 and populates a corresponding view in the secondary storage system208. In one embodiment, view 210, 212, and 214 generally corresponds toa backup state of one of the primary systems as data is retrieved fromthat primary system and written to the secondary storage system. Viewsprovide backup data from a primary system “frozen” for a particularpoint in time, accordingly, a view may be referred to as a “snapshot.”Thus, a snapshot refers to a consistent backup state for a primarysystem created at a specific point in time. For example, a snapshot of acurrent view of may be created of every five minutes. To do so, a newsnaptree root node is created for each new snapshot every five minutes.

To create a new snapshot efficiently, the secondary storage system 300may need only to create a copy of the root node of view from which thenew snapshot is being created. Thus, when created, a new snapshot hasits own snaptree root node, but this root node points back into thesnaptree of the view from which the new snapshot is created. Thereafter,when a write is made to the new snapshot, new nodes in the new snaptreeare created to store the modified data. Doing so allows the new snaptreeto manage a current backup state (i.e., to provide the current view),while allowing prior backup states (accessed via the root node thesnaptree being cloned) to point to prior versions of the same data.Accordingly, each view may have some nodes local that view but refer tonodes of other prior views where data has not changed. Similarly, a rootnode of a view for a newly created snapshot (also referred to as a clonein this context) may be created to expose a view to a non-productionworkload. In doing so, any writing done by this workload is reflected innodes of the clone, while referring back to other views for data that isnot changed by performing the non-production workload.

FIG. 3 depicts software processes in a secondary storage system 300 forcollecting and externally exposing views, according to one embodiment.As shown, the software processes include a bridge process 322 and a datagathering process 302. Additionally, the bridge process 322 includes aportal management process 344 for exposing data from the secondarystorage system to external systems running the non-production workloads,a binary large object (BLOB) manager 342, and a scribe process 340. Thescribe process 340 manages and stores data and metadata for thesecondary storage system 300. That is, the scribe process 340 isgenerally configured to write and read to the key-value store on a givennode. Other processes on the secondary storage server 300 ensure thatchanges made on one node are replicated to other nodes of a cluster ofsecondary storage systems 300.

In one embodiment, the data gathering process 302 provides an interfacebetween the secondary storage server 300 and external data sources(e.g., a data store of one of the primary systems 348 and 346). When thesecondary storage server 300 accesses a primary system, a protocolspecific to that primary system is typically used to fetch data. Forexample, as shown, the primary system 348 uses the RMAN protocol tofetch data from an primary system running an Oracle database and primarysystem 346 uses the VADP APIs to fetch data from a primary systemrunning a VMware cluster. Of course, depending on the primary systembeing backed up by the secondary storage server 300, other protocols forbacking up data may be used.

As data is captured from a primary system (e.g., one of primary systems346 and 348), the data gathering process 302 may store the backup data324 in a current view 328 corresponding to that primary system.Additional views of the backup data 324 may correspond to incrementalbackups of the primary system data relative to specific points in time.For example, FIG. 3 shows views 326, 328, 330, and 334. In this example,the current view 328 generally corresponds to a “live” or most currentbackup state of data captured from a given primary system. Other views(e.g., views 326, 330, 334) can be derived from the current view 328 (orfrom other views). As shown, e.g., view 334 may provide a backup of thecurrent view 328 captured at a particular point in time. In oneembodiment, the secondary storage system 300 may create an incrementalbackup every five minutes (or other at other regular intervals orspecified times), where each incremental backup captures a backup stateof data of the primary system at the time the incremental backup iscreated. Additional examples of creating a view of backup data 324 aredescribed below. Views 326 and 330 may be views created to expose a setof backup data to external system 350. Such views may provide aread-write clone of one of the incremental backups (e.g., of view 334).

When an external system 350 is used to access data 324 in secondarystorage system 300, the external system 350 may do so by accessing dataexposed by a particular view. Further, a view may be specificallycreated for a given external system 350. Doing so effectively gives theexternal system a “private” copy of the backup data, without having toactually replicate a complete copy of the backup data. In oneembodiment, any modifications made to such a view are visible only tothat view and therefore only to the external system 350. Further, once abackup of a primary system is captured, the secondary storage system 300may expose views of the backup data to external systems 350 runningnon-production workloads using a variety of protocols. For example, asshown in FIG. 3, the portal management process 344 may expose data 324using server message block (SMB) 312, network file system (NFS) 314,Internet Small Computer System Interface (iSCSI) 316, and the HadoopDistributed File System (HDFS) 320. In addition to exposing data usingthese (or other) file system protocols, the portal management process344 may also expose data 324 using other representations or formats,e.g., as objects based on the representational state transfer (REST)protocol. Of course, a view of backup data stored by the secondarystorage system could be exposed to the external system 350 using avariety of other protocols, formats, or data representations.

Views created for an external system 350 can be persisted or discardedas needed for a particular case. Further, the variety of interfacesallow the secondary storage system 300 to maintain an internalorganization for the snapshot data that is independent from the dataorganization implied by any of the protocols used to present the backupdata to an external system 350. For example, if objects in the primarysystem are backed-up, then an external system may access those objectsthrough the REST interface 318, without requiring that the secondarystorage system to store the object data as objects.

Additionally, the secondary storage system 300 may maintain distinctnamespaces for any given view. FIG. 4 depicts an example of variousnamespaces, according to one embodiment. In an embodiment, thenamespaces include a file namespace 402, an object namespace 404, and aconfiguration namespace 406. The file namespace 402 is used forcapturing data in the form of files from the primary system. The objectnamespace 404 is used to store information related to objects backed upfrom a primary system. For example, a primary system might maintain datain the form of objects, e.g., the Openstack platform includes acomponent named Swift that maintains objects. If the secondary storagesystem is used to backup objects from the Swift component, the objectnamespace 4040 may store namespace information for such objects. Theconfiguration namespace 406 is used to capture configuration informationfrom the primary system. As noted, each view 402, 404, 406 may beassociated with a snaptree, some nodes of which are depicted as nodes410 412, 414.

FIG. 5 depicts a process 500 for creating a view, according to oneembodiment. As shown, process 500 begins at step 502 where the secondarystorage system receives data backup parameters for backing up a primarysystem. Such parameters may include bandwidth limitations on obtainingdata from the primary system and a backup schedule that includes thefrequency of performing the backup. Some parameters may be specific tothe primary system being backed up. For example, to backup a VMwarecluster, the IP address and credentials to access the VMware's vCentercomponent that manages that cluster are needed. Once configured, thedata gathering process may begin transferring data from the primary tothe secondary storage system (step 504). As the data streams into thesecondary storage system, the data is written to a current viewcorresponding to that primary system (at step 506). Further, otherviews, or snapshots, of the current view may be created at any time.

FIG. 6 depicts a process for creating a snapshot, according to oneembodiment. As noted, a snapshot generally “freezes” a backup state fora primary system at a specific point in time. Alternatively, a snapshotmay create a clone of a existing view (i.e., of an incremental backup).As shown, at step 602, the secondary storage system determines that asnapshot should be created. For example, users may manually create asnapshot of a view to provide a data store for a secondary ornon-production system. Similarly, as the secondary storage system isused to backup a primary storage system, the secondary storage systemmay create snapshots of the current view of that system at regularintervals—allowing the primary storage system to be restored tovirtually any prior state. For example, the secondary storage system maycreate a new snapshot of a current view of a primary system every 5minutes. In this context, each view provides a restore point for theprimary storage system.

At step 604, the secondary storage system creates a new snapshot asneeded in a particular case. In one embodiment, the secondary storagesystem creates a new snapshot of a view by creating a copy of the rootnode of the view from which the snap snapshot is being made. The cloneof the root node includes pointers that reference the same child nodesas the root node of the view being cloned.

Further, once created, each view (i.e., each snapshot) may be accessedby a variety of other processes and systems. For example, as noted, adeveloper may access a view (or create a clone) to provide productiondata for a test and development system or a view may be exposed to adata analytics system. Similarly, each view could be traversed by adisaster recovery or archival process, allowing a backup (of the backupprovided by the secondary storage system) to be replicated to a disasterrecovery site or periodically written to a cloud-based archival service.For example, if a developer needs a local or private copy of aproduction data store, the developer can create a snapshot of that datastore from any view created by the secondary storage system (such as oneof the views created automatically every five minutes). To do so, thesystem creates a copy of the root node of the snaptree of the desiredview and exposes a view of the cloned root node to the developer in arequested format (e.g., as an NFS mount point). Because the cloned rootnode points to the same child nodes as the root node of the snaptreebeing cloned, any reads of the cloned snaptree simply “reach back” intothe snaptree being cloned. At the same time, any writes made to thecloned snaptree result in new branches in the cloned snaptree that donot “reach” back into the original. Instead, nodes written in the clonedroot node are accessible only via that cloned root node.

Similarly, for the “current view” of a primary system, an incrementalbackup is created by copying the root node of the current view—whichthen becomes the current view for subsequent writes made by the datagathering process. That is, the root node of the current view (the onebeing copied) becomes the root node of the incremental backup. Moregenerally, should any view need to remain writeable, any snapshot ofthat view results in two copies of the root node being created. Doing soallows the view to remain writeable by performing subsequent writes toone of the copies of the root node, while preserving the state of thatview existing at the time the snapshot is made. That is, one clone ofthe root node is used for the snapshot (the snapshot clone) and theother for any subsequent writes to that view from which the snapshot istaken (the write clone). Doing so allows for read operations madeagainst either the snapshot clone or write clone to “reach back” intothe original view. At the same time, write operations are performed toeither the write clone or the snapshot clone (depending on which view isbeing written to, the original view or the snapshot).

In one embodiment, the scribe process 340 in FIG. 3 manages a key valuestore ((k,v)-store), which includes a plurality of (k,v) pairs, where kis a key and v is a value associated with the key. The (k,v) pairs areavailable from the (k,v) store. In the secondary storage system, the(k,v)-store is used to store data structures for handling data backupsgenerated for the primary storage system, as well as to expose backupdata (as a snapshot) to non-production systems. In one embodiment, the(k,v)-store managed by the scribe process is stored in the SSD storage112, 118 in FIG. 1, so that the scribe process can quickly access andupdate the key-value pairs.

As noted, snapshots described above are stored in tree data structure,referred to as a snaptree. In one embodiment, leaves of the a snaptreestore a key-value pair the (k,v)-store. That is, leaves of the snaptreestore metadata indicating where data (v) corresponding to a key (k) maybe found (or a copy of the data corresponding to a given key). Nodesthat are not leaf nodes have pointers to child nodes. FIG. 7 depicts asnaptree formed from (k,v) pairs managed by scribe, according to oneembodiment. In the figure, the root 702 of the snap tree has children704 and 710. Child node 704 has children 706 and 708. Child node 710 haschildren 712 and 714. Each leaf node includes a (k,v) pair as shown.Other nodes indicate the range of key values may be accessed bytraversing to a next node using a given pointer. In one embodiment, aleaf also may include file system data, when that data under a givensize. That is, in some cases, the snaptree—generally used to storemetadata indicating where data associated with a key value (k) may befound—stores the actual data associated with a given key. Note, the sizeof data stored in the snaptree directly (and typically therefore in SSDstorage), may be tailored as a matter of preference. However, a value of256 Kbytes has proven to be effective. Otherwise, the values in eachkey-value pair leaf point to a BLOB storing data corresponding to thekey

Using snaptrees for snapshots provides several advantages with respectto both searching and modifying snapshot data, and with respect toisolation of workloads. The snaptree can be searched efficiently andrapidly to find data because searching a tree is bounded by the depth ofthe tree, which is much less than if the nodes were arranged in linearorder. The snaptree can also be modified quickly. FIG. 7 depicts anincremental modification 716 to the snaptree 702, according to oneembodiment. As shown, child nodes 710 and 714 are modified by anincrement 716, which is a new root to an independent tree 716, 710′,720′ involving nodes 704, 710, 712, and 714 as indicated by markers 722,724, and 726. To handle an incremental modification, the nodes 710 and714 are copied to nodes 710′ and 720′ of the new tree and then modified.This is efficient because only the nodes 710′ 720′ involved in themodification are copied and a search for the modified data is againbounded by the depth of the new tree. Finally, the snaptree providesisolation with respect to the data it controls, because the system canmaintain separate snaptrees for data involved in different workloads.

FIG. 8 depicts another view of a snaptree 802 and of BLOB trees 810,814, according to one embodiment. As mentioned, the view data iscaptured and stored in snaptrees. When the size of the data isrelatively small, the data is stored directly in the snaptree 802 alongwith metadata necessary to find the actual data. When the size of thedata is relatively large, the snaptree 802 stores metadata, which pointsto a BLOB 810, 814. The BLOB 810, 814 in turn has nodes that describecontainers called bricks 812,816. In one embodiment, a brick 812, 816 isa one megabyte block of data. Bricks in turn contain chunks 822, 824,which are smaller containers for data. A chunk in a BLOB may or may notbe deduplicated. If a chunk is deduplicated it may not align withbricks. If a chunk is not deduplicated, it aligns with a brick. A chunk822, 824 points to a chunk table 826, whose contents contain pointers tofile ids 828. The pointer 828 to file IDs points to a chunk file table830 that contains the portion of data sought.

FIG. 9 depicts a process 900 for accessing a snaptree to service a datarequest, according to one embodiment. For example, a user may beinteracting with system used to perform test and development functionsor to perform data analytics using data stored on the secondary storagesystem. For example, a user doing application testing and developmentmay want to test a new (or new version of an) enterprise applicationusing a snapshot of actual production data. To do so, the user selectsthe desired snapshot, which may be exposed as needed for the enterpriseapplication being tested (e.g., as an NFS mount point). Once mounted,the secondary storage system allows the test application to access datain the snapshot as though it were a block storage device in the case ofan access via iSCSI or as a file system in the case of an access viaNFS. As shown, the process 900 begins at step 902 where the bridgeprocess 322 in FIG. 3 receives an interface request, such as an NFSrequest, for secondary storage system data. In response, at step 904,the bridge process derives a logical key from the parameters in the NFSrequest. For example, the parameters of the NFS request could include aninode ID, a viewID, and a range of data. At step 906, the bridge processthen uses the logical key to look up the root of the snaptree containingthe data (or metadata) for the request.

At step 906, bridge process 322 then traverses the snaptree to find anindex node (inode) that indicates the location of the data. If theamount of data attached to the inode is less than LIMIT, (e.g., LIMIT is256K bytes), then the data is stored in the value of the key as part ofthe node of the snaptree itself along with the inode. If so, the bridgeprocess returns the data at step 910. If at step 908, the size of therequest is large then the inode points to a binary large object (BLOB),which may provide another snaptree containing further information aboutthe location of the requested data. In such case, the bridge processobtains and ID for a BLOB (step 912). In step 914, the bridge processthen searches the BLOB to find the data. The node or leaf for therequested data contains an item called a brick, which in one embodimentis 1 Megabyte. The brick object contains one or more chunks and thechuck for the requested data is then found in step 918. The chunk thenleads to a chunk table in step 920, which contains a chunk file id andthe chunk file ID then leads to a chunk file table, which finally givesthe location of the requested data, which is then returned to satisfythe request in step 922.

In this manner, snaptrees provide rapid access to requested data,including rapid access to random items of data in the snaptree.Additionally, because increments to the snaptree data are stored in atree-like manner, read and write access to the increment data is alsorapid. Finally, because each different type of data is stored indifferent snaptrees, workload traffic to one snaptree has very littleeffect on workload access to a different snaptree. This allows thesecondary storage system to sustain widely different types of workloadsincluding normal backup workloads along with non-production workloads,such as business analytics and test and development workloads that wereserviced by separate systems.

FIG. 10 depicts an example computing system 1000 for a secondary storageserver. As shown, the computing system 1000 includes, withoutlimitation, a central processing unit CPU 1002, a network interface1006, a memory 1008 for storing CPU programming instructions and storage1010, each connected to a bus 1016. The computing system 1000 may alsoinclude an I/O device interface 1004 connecting I/O devices (e.g.,keyboard, display, and mouse devices) to the computing system 1000.Further, in context of this disclosure, the computing elements shown incomputing system 1000 may correspond to a physical computing system(e.g., a system in a data center) or may be a virtual computinginstance.

The CPU 1002 retrieves and executes programming instructions stored inthe memory 1008 as well as stores and retrieves application dataresiding in storage 1010. The memory 1008 also stores the views 1016,1018, the bridge processes 1020 and the data gathering processes 1024for carrying out the functions of the secondary storage system. The bus1016 is used to transmit programming instructions and application databetween the CPU 1002, I/O devices interface 1004, storage 1010, networkinterface 1006, and memory 1008. Note, CPU 1002 is included to berepresentative of a single CPU, multiple CPUs, a single CPU havingmultiple processing cores, and the like. The memory 1008 is generallyincluded to be representative of a random access memory. The storage1010 may include a disk drive storage device 1014 and a solid statestorage device 1012. In one embodiment, the disk drive storage device1014 stores backup and non-production data and the solid state storagedevice 1012 stores snaptrees 1028, 1030. Although shown as a singleunit, the storage 1010 may be a combination of fixed and/or removablestorage devices, such as fixed disc drives, removable memory cards,optical storage, network attached storage (NAS), or a storagearea-network (SAN).

FIG. 11 depicts an arrangement of a production system hosting one ormore enterprise applications, a test and development system, and asecondary storage system according to one embodiment. The productionsystem 1104 with the enterprise applications 1110 and production systemdata 1112 is connected to a network 1102, along with test anddevelopment systems 1108 and a secondary storage system 1106 withsecondary storage data 1114, 1116. The network 1102 can be a local areanetwork, a wide area network, or the Internet or combinations thereof.The secondary storage system provides both backup and non-productiondata services for the production system 1104 and the test anddevelopment systems 1108.

Aspects of the present invention may be embodied as a system, method orcomputer program product. Accordingly, aspects of the present inventionmay take the form of an entirely hardware embodiment, an entirelysoftware embodiment (including firmware, resident software, micro-code,etc.) or an embodiment combining software and hardware aspects that mayall generally be referred to herein as a “circuit,” “module” or“system.” Furthermore, aspects of the present invention may take theform of a computer program product embodied in one or more computerreadable medium(s) having computer readable program code embodiedthereon.

Any combination of one or more computer readable medium(s) may beutilized. The computer readable medium may be a computer readable signalmedium or a computer readable storage medium. A computer readablestorage medium may be, for example, but not limited to, an electronic,magnetic, optical, electromagnetic, infrared, or semiconductor system,apparatus, or device, or any suitable combination of the foregoing. Morespecific examples a computer readable storage medium include: anelectrical connection having one or more wires, a portable computerdiskette, a hard disk, a random access memory (RAM), a read-only memory(ROM), an erasable programmable read-only memory (EPROM or Flashmemory), an optical fiber, a portable compact disc read-only memory(CD-ROM), an optical storage device, a magnetic storage device, or anysuitable combination of the foregoing. In the current context, acomputer readable storage medium may be any tangible medium that cancontain, or store a program for use by or in connection with aninstruction execution system, apparatus or device.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality and operation of possible implementations ofsystems, methods and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment or portionof code, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations the functions noted in the block may occur out of theorder noted in the figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. Each block of the block diagrams and/orflowchart illustrations, and combinations of blocks in the blockdiagrams and/or flowchart illustrations can be implemented byspecial-purpose hardware-based systems that perform the specifiedfunctions or acts, or combinations of special purpose hardware andcomputer instructions.

The foregoing description, for purpose of explanation, has beendescribed with reference to specific embodiments. However, theillustrative discussions above are not intended to be exhaustive or tolimit the invention to the precise forms disclosed. Many modificationsand variations are possible in view of the above teachings. Theembodiments were chosen and described in order to best explain theprinciples of the invention and its practical applications, to therebyenable others skilled in the art to best utilize the invention andvarious embodiments with various modifications as may be suited to theparticular use contemplated.

While the foregoing is directed to embodiments of the present invention,other and further embodiments of the invention may be devised withoutdeparting from the basic scope thereof, and the scope thereof isdetermined by the claims that follow.

What is claimed is:
 1. A method, comprising: receiving a first set ofdata from a primary storage system to store in a secondary storagesystem, wherein the received data provides a backup of data stored bythe primary storage system, wherein the first set of data corresponds toa first snapshot of the primary storage system; generating at least afirst view of the first set of data corresponding to the first snapshotin the secondary storage system, wherein the first view of the first setof data corresponding to the first snapshot includes a first tree datastructure having a root node, one or more intermediate nodes, and one ormore leaf nodes; receiving a second set of data from the primary storagesystem to store in the secondary storage system, wherein the second setof data corresponds to a second snapshot of the primary storage system;generating, from the first view of the first set of data correspondingto the first snapshot, at least a second view of the second set of datacorresponding to the second snapshot in the secondary storage system,wherein generating the second view corresponding to the second snapshotcomprises copying the root node of the first tree data structureassociated with the first view of the first set of data corresponding tothe first snapshot, wherein the second view of the second set of datacorresponding to the second snapshot includes a second tree datastructure that at least includes the copied root node, wherein thecopied root node includes one or more corresponding pointers to the oneor more intermediate nodes of the first view of the first set of datacorresponding to the first snapshot; and in response to the second setof data including new data, generating one or more new leaf nodescorresponding to the new data in the second tree data structureassociated with the second view of the second set of data correspondingto the second snapshot, wherein the copied root node associated with thesecond view of the second set of data corresponding to the secondsnapshot includes a pointer that directly or indirectly references theone or more new leaf nodes, and wherein the one or more new leaf nodesare not exposed by the first view of the first set of data correspondingto the first snapshot.
 2. The method of claim 1, wherein the receivedfirst set of data provides a backup of data stored by the primarystorage system.
 3. The method of claim 1, wherein the one or more leafnodes and the one or more new leaf nodes have a corresponding key-valuepair that indicates a location of corresponding data in the secondarystorage system.
 4. The method of claim 1, wherein the first set of datacorresponds to the first snapshot of the primary storage system at afirst moment in time.
 5. The method of claim 4, wherein the second setof data corresponds to the second snapshot of the primary storage systemat a second moment in time, wherein the first moment in time occurredbefore the second moment in time.
 6. The method of claim 1, whereingenerating one or more new leaf nodes corresponding to the new data inthe second tree data structure associated with the second view of thesecond set of data corresponding to the second snapshot comprises:generating an intermediate node; updating one of the one or morecorresponding pointers that points to one of the one or moreintermediate nodes of the first view of the first set of datacorresponding to the first snapshot to point to the generatedintermediate node.
 7. The method of claim 6, wherein the generatedintermediate node is a copy of the one of the one or more intermediatenodes of the first view of the first set of data corresponding to thefirst snapshot.
 8. The method of claim 6, wherein the intermediate nodeincludes a pointer that points to one of the one or more new leaf nodes.9. The method of claim 1, further comprising: receiving a request fordata exposed by the second view of the second set of data correspondingto the second snapshot; traversing the second tree data structure fromthe copied root node to one of the one or more new leaf nodes; andreturning the requested data.
 10. The method of claim 1, furthercomprising: receiving a request for data exposed by the second view ofthe second set of data corresponding to the second snapshot; traversingthe second tree data structure from the copied root node to one of theone or more leaf nodes associated with the first tree data structure,wherein the copied root node includes a pointer that directly orindirectly references the one of the one or more leaf nodes associatedwith the first tree data structure.
 11. The method of claim 1, whereinthe one or more leaf nodes include file system data.
 12. The method ofclaim 1, wherein at least one of the one or more leaf nodes includes apointer to a third tree data structure.
 13. The method of claim 12,wherein a leaf node of the third tree data structure includes a pointerto a chunk file table.
 14. The method of claim 1, wherein at least oneof the one or more new leaf nodes includes a pointer to a third treedata structure.
 15. A system, comprising: a processor configured to:receive a first set of data from a primary storage system to store in asecondary storage system, wherein the received data provides a backup ofdata stored by the primary storage system, wherein the first set of datacorresponds to a first snapshot of the primary storage system; generateat least a first view of the first set of data corresponding to thefirst snapshot in the secondary storage system, wherein the first viewof the first set of data corresponding to the first snapshot includes afirst tree data structure having a root node, one or more intermediatenodes, and one or more leaf nodes; receive a second set of data from theprimary storage system to store in the secondary storage system, whereinthe second set of data corresponds to a second snapshot of the primarystorage system; generate, from the first view of the first set of datacorresponding to the first snapshot, at least a second view of thesecond set of data corresponding to the second snapshot in the secondarystorage system, wherein generating the second view corresponding to thesecond snapshot comprises copying the root node of the first tree datastructure associated with the first view of the first set of datacorresponding to the first snapshot, wherein the second view of thesecond set of data corresponding to the second snapshot includes asecond tree data structure that at least includes the copied root node,wherein the copied root node includes one or more corresponding pointersto the one or more intermediate nodes of the first view of the first setof data corresponding to the first snapshot; and in response to thesecond set of data including new data, generate one or more new leafnodes corresponding to the new data in the second tree data structureassociated with the second view of the second set of data correspondingto the second snapshot, wherein the copied root node associated with thesecond view of the second set of data corresponding to the secondsnapshot includes a pointer that directly or indirectly references theone or more new leaf nodes, and wherein the one or more new leaf nodesare not exposed by the first view of the first set of data correspondingto the first snapshot; and a memory coupled to the processor andconfigured to provide the processor with instructions.
 16. The system ofclaim 15, wherein the received first set of data provides a backup ofdata stored by the primary storage system.
 17. The system of claim 15,wherein the one or more leaf nodes and the one or more new leaf nodeshave a corresponding key-value pair that indicates a location ofcorresponding data in the secondary storage system.
 18. The system ofclaim 15, wherein the first set of data corresponds to the firstsnapshot of the primary storage system at a first moment in time. 19.The system of claim 18, wherein the second set of data corresponds tothe second snapshot of the primary storage system at a second moment intime, wherein the first moment in time occurred before the second momentin time.
 20. A computer program product, the computer program productbeing embodied in a non-transitory computer readable storage medium andcomprising computer instructions for: receiving a first set of data froma primary storage system to store in a secondary storage system, whereinthe received data provides a backup of data stored by the primarystorage system, wherein the first set of data corresponds to a firstsnapshot of the primary storage system; generating at least a first viewof the first set of data corresponding to the first snapshot in thesecondary storage system, wherein the first view of the first set ofdata corresponding to the first snapshot includes a first tree datastructure having a root node, one or more intermediate nodes, and one ormore leaf nodes; receiving a second set of data from the primary storagesystem to store in the secondary storage system, wherein the second setof data corresponds to a second snapshot of the primary storage system;generating, from the first view of the first set of data correspondingto the first snapshot, at least a second view of the second set of datacorresponding to the second snapshot in the secondary storage system,wherein generating the second view corresponding to the second snapshotcomprises copying the root node of the first tree data structureassociated with the first view of the first set of data corresponding tothe first snapshot, wherein the second view of the second set of datacorresponding to the second snapshot includes a second tree datastructure that at least includes the copied root node, wherein thecopied root node includes one or more corresponding pointers to the oneor more intermediate nodes of the first view of the first set of datacorresponding to the first snapshot; and in response to the second setof data including new data, generating one or more new leaf nodescorresponding to the new data in the second tree data structureassociated with the second view of the second set of data correspondingto the second snapshot, wherein the copied root node associated with thesecond view of the second set of data corresponding to the secondsnapshot includes a pointer that directly or indirectly references theone or more new leaf nodes, and wherein the one or more new leaf nodesare not exposed by the first view of the first set of data correspondingto the first snapshot.